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IMPROVEMENTS IN OR RELATING TO IJ^AGE PROCESSING 

The present invention relates to image processing and, in particular, to the 
enhancement of images to assist in their interpretation. : . 

5 There are many techniques for the processing of images, particularly digitised 

images, to reduce noise and assist in their interpretation. Such techniques are particularly 
important in the field of medical imaging, where images are typically noisy and diJB&cult ' 
to interpret since clinically significant signs are mostly subtle. As an example, x-ray 
imaging is used as a basis for rnany medical techniques and, in particular, 

1 0 mammography is currently the examination of choice for early detection of breast 
cancer. One of the earliest indicators of breast cancer is the presence of 
microcalcifications, which can often be identified in mammograms as localized bright 
spots. In the accompanying drawings Figure 1 illustrates some mammogram samples 
showing microcalcifications. Figures 1 A and B show isolated calcifications, while 

1 5 Figures 1 C and D show microcalcification clusters. It woxild be usefiil to have image 
enhancement techniques which assist radiologists or clinicians in fin d ing 
nGdcrocalcifications in mammogram images. However, it is important that such 
techniques miss as few clinically important microcalcification clusters as possible, and 
also do not signal too many false positives. 

20 Methods of detecting automatically microcalcifications in mammograms have 

• been proposed, for instance in WO-A-00/52641 and WO-A-01/69533. These techniques 
are beised on an adapted version of the image known as the representation in which 
the specific imaging parameters particular to the imaging process are removed. This 
results, in essence, in a normalised image known as the Standard Mammogram Format 

25 (SMF) which can be displayed as an h^^ surface, or witht the hi„, values converted into 
grey levels, in which case the image resembles a conventional mammogram. The 
techniques for producing the h^^ representation will not be repeated here, but they are 
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explained in detail in WO-A-00/52641 which is incorporated herein by reference. 
Figures 2 and 3 of the accompanying drawings illustrate respectively an original 
mammogram and the different or SMF representations. In Figure 2 the values are 
shown as a siuf ace whereas in Figure 3 the hf^ values are converted into grey levels and 
5 displayed as an SMF akin to. a conventional mammogram. 

One technique for enhancing images is known as "diffusion". This is, in essence, 
a smoothing process in which the image is processed by convolving the intensity values 
in the image with a kernel for instance a Gaussian kernel. Although such a smoothing 
process can assist in enhancing images, it can also create problems. In particular, in an 

10 image containing an object shown against a background, smoothing or blurring of the 
object into the background is undesirable. Therefore so called "anisotropic diffusion" 
techniques have been proposed in which the diffusion processing occurs within objects, 
and within the background, but not across the boundaries between the two. Such 
techniques are disclosed, for instance, in "Scale-Space and Edge Detection Using 

15 Anisotropic Diffusion" by Perona and MaUk (IEEE Transactions on Pattern Analysis 
and Machine Intelligence, volume 12, number 7, July 1990) and "Robust Anisotropic 
Diflftision" by Black et al. (IEEE Transactions on Image Processing, volvime 7, number 
3, March 1998) which are incorporated herein by reference. In these techniques, though, 
flie parameters of the diffusion process which include the number of iterations, the scale 

20 of the kernel and the diffusion.coeflBcient itself, are typically set interactively by the 

user. This would be impractical for medical image processing in which many images of 
different qualities and characteristics are produced, thus requiring the automatic setting 
of parameters for each individual image. 

The first aspect of the present invention provides a method of processing 

25 mammogram images using an anisotropic diffusion process in which the anisotropic 
diffusion process is adaptive in dependence upon the image being processed. The 
process adapts itself in accordance with the characteristics of the image, eg a measure of 
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the contrast in tiie image. This adaptation is automatic, and thus does not require the user 
to assess the image and set the difiusion process parameters for each different lype of 
image. 

The diffusion process may be made adaptive by changing its parameters, e.g. its 
5 difiusion coefficient, and/or, for example, the number of iterations in the process. For 
example, the diffusion coefficient may be dependent upon the contrast in the image. In 
particular it may be calculated fiom a statistical analysis or measure of the local contrast 
in the image, e.g. based on an average value, and standard deviation of the local contrast 
values. 

10 Thus the invention involves taking an original image, possibly processing it to 

enhance it using known techniques, such as in a mammogram to produce the Standard 
M amm ogram Format, possibly performing other enhancements such as taking the 
Gaussian derivative, and then applying an anisotropic difiusion process in which at least 
one of the parameters of the diffusion process are calculated from the characteristics of 

1 5 this particular image. 

It is found that this technique allows the appUcation of anisotropic difiusion 
processing to many different images, e.jg. different mammograms, on an automated 
basis. In the case of mammograms it provides enhancement of the visibiUty in the 
processed image of microcalcifications. 

20 Another aspect of the invention provides a method of segmenting an object in an 

image firom the background of the image by using a contrast based segmentation 
method, such as a so-called foveal segmentation algorithm, in which the segmentation 
algorithm is made adaptive by being dependent upon the characteristics of the image 
being processed, such as the contrast. 

25 Foveal segmentation is a segmentation based on the local contrast in areas of the 

image. It is based on an analysis of human brightness perception as explained in "A New 
Image Segmentation Method Based on Hxraian Brightness Perception and Foveal 
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Adaptation" by Heucke et al (IEEE Signal Processing Letters, volume 7, number 6, June 
2000), In flie technique described in that paper, areas of an image are assigned to belong 
to either an object or the background depending on whether the local contrast is above a 
certain minimum contrast. The mirn'mum contrast is calculated to be the mi'niTmim 
5 contrast perceivable by the human eye. With this aspect of the present invention, 

however, the segmentation technique is developed so that at least one of the parameters 
of the segmentation process is calculated firoin the image characteristics. This allows the 
automatic segmentation of images of different characteristics without the need for the 
user interactively to set the segmentation parameters. Conveniently the mmirmiTn 
10 contrast value is deJBned with respect to the contrast in the image, for instance a 

statistical analysis or measure of the local contrast in the image, e.g. based on an average, 
value and standard deviation of the local contrast values. 

The two aspects of the invention may be combined together and they are 
particularly useful for processing medical x-ray images, particularly digitised 
15 mammograms. 

The invention also extends to a computer program comprising program code 
means for executing liie image processing method on a suitably jprogrammed computer 
system, to a computer readable storage medium carrying the computer program, and to 
an image processing apparatus for executing the image processing method. 
20 The invention will be further described by way of example with reference to the 

accompanying drawings in which:- 

Figures 1 A to D illustrate mammograms including microcalcifications; 

Figures 2A and B illustrate respectively a mammogram and its Standard 
Mammogram Format; 
25 Figures 3 A and B illustrate a mammogram and its Standard Mammogram 

Format; 

Figure 4 is a flow diagram illxistrating image processing according to one 



embodiment of the present invention; 

Figure 5A shows an example of a mammogram containing aiarge calcification 
and several artifacts and Figure 5B illustrates the result of dtfiEusing the image of Figure 
5A; 

5 Figure 6A illustrates an SMF image containing a microcalcification. Figure 6B 

the diffused conversion of image A, Figure 6C a 3-D plot of the SMF image of Figure 
6A, and Figure 6D the surface of the diffused image in Figure 6B; 

Figures 7A through D illustrate the removal of artifact from a mammogram, 
namely successively: Figure7A illustrates the original knage. Figure 7B a map of the 
10 shot-noise. Figure 7C a map of curvilinear structures ia the image and Figure 7D the 
image after shot-noise and curvilinear structure removal; 

Figure 8 illustrates an original image and diffused versions of the image with 
different parameters; 

Figure 9 illustrates an original mammogram and its surface plot together with 
15 diffused versions of the mammogram and surface plot; 

Figures lOA and B illustrate original images with calcifications. Figures IOC and 
D illustrate gradient maps for the images of Figures lOA and B and Figures IDE and F 
illustrate diffused versions of the images of Figures 1 OA and B; 

Figures 1 1 A through J illustrate original SMF images alongside corresponding 
20 processed images in which the microcalcificati.ons have been detected and marked in 
accordance with an embodiment of this invention. 

' Figure 4 is a flow diagram of image processing in accordance with one 
embodiment of the invention. In this embodiment the processing according to the 
25 iavention is preceded by processing which has previously been proposed to enhance the 
image. 

The grey-level original image 1 is first blurred using a Wiener filter. This de- ' 
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noises the image to an extent by removing radiographic mottle, which is a source of false 
positives in detectiiig microcalcifications..The Wiener filter is adapted to the 
characteristics of radiographic noise in the Original image. This technique is es^plained in 
Yam, M. Brady, J.M. Highnam, R.P. English, R.: Denoising h^^ Surfaces: a Physics- 
5 based Approach, in Medical Image Computing and Computer- Assisted Intervention 
1999, Springer- Verlag, Berlin Heidelberg New York (1999) 227-234, incorporated 
herein by reference. * 

The next step is the generation of the Standard Mammogram Format (SMF) 5 
using the technique described in WO-A-00/52641 . 

1 0 This may be further processed by the glare removal technique disclosed in WO- 

A-00/52641 to produce the blurred, no glare, SMF 7. 

A major source of errors in detecting microcalcifications is film-screen shot 
noise, which appears primarily from small pieces of dust on the intensifying screen and 
has visual properties which are similar to those of microcalcifications. However, because 

1 5 shot noise is caused, for example, by dust on the screen rather tiian by structures within 
the breast, it is characterised by the absence of blur. Therefore such shot noise may be 
detected by the absence of blur, and then removed from the image. This technique is 
described in Highnam, R.P. Brady, J.M. English, R.: Detecting Fihn-Screen Artifacts in 
Mammography using a Model-Based Approach, in IEEE Transactions in Medical 

20 Imaging, Vol. 18 (1999) 1016-1024 which is herein incorporated by reference. Further, 
curvilinear structures in the breast have similar visual properties to microcalcifications 
when viewed in a noisy image. It is advantageous, therefore, to use one of the available 
techniques for the removal of curvilinear structures, for example based on phase 
congraency as disclosed in Yates, K. Evans, C.J. Brady, J.M. : Improving the Brake's 

25 Manamographic Mass Detection Algorithm Using Phase Congruency, in Proceedings of 
Digital Image Computing: Techniques and Applications, Melbourne (2002), which is 
herein incorporated by reference. 
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This results in an enhanced SMF 9. Figure 7 illustrates this artifact removal 
process. Figure 7A illustrates the original image and Figure 7B the shot-noise map 
(white dots are noise). Figure 7C illustrates the curvilinear structure map^ and Figure 7D 
the "clean" image after shot-noise and CLS removal. 
5 Next, in accordance with the invention, the clean SMF 9 is subjected to an 

adaptive anisotropic diffusion process to produce a diffused image 11, and then to 
adaptive foveal segmentation to produce a map of microcalcifications 13. These * 
processes are rendered adaptive by using a parameter k which is representative of the 
local contrast in the image. This parameter is derived from a gradient map 15 whose 

1 0 calculation will be described below. 

The parametric format of anisotropic diffusion makes it highly dependent upon 
the fine-tuning of its input parameters. There are three parameters to be considered when 
attempting to blur an image using anisotropic diffusion: k - the contrast, / - the time or 
number of iteratiorus and g - the standard deviation of scale. In practice, the more 

1 5 complex and variable the image is in a data set, the more problematic it is to choose a 
single set of values for these parameters that work well for the entire data set Medical 
images, and certainly mammograms, are very complex images whose appearance varies 
widely across a population (at a centre, hospital, region, country or continent), which 
makes the vital requirements of generating few false positives and fewer false negatives 

20 very difQctdt. 

In accordance with this embodiment of the present invention the contrast 
parameter k^ which is image dependent, is varied in dependency upon the characteristics 
of the image. The time parameter t is set to be constant (i.e. a constant number of 
iterations), as is the scale. 

25 In accordance with this embodiment of the invention the adaptive anisotropic 

diffusion is conducted with parameters, in particular a contrast value, derived from use 
of a Gaussian derivative filter. Firstly, the SMF 9 is processed to derive the Gaussian 
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derivative of flxe image in accordance with equations 2 and 3 below:- 



■ w 



2 \ 



(2) 



^=K'(/)| (3) • 

5 where K is the Gaussian of image 7 and the Gaussian derivative. 

Then the values of the local contrast are calculated for the image in accordance 
with equation 4 helow:- . ' ■ 




The local contrast is calculated in a neighbourhood of N pixels. These values may be 
displayed in a gradient map as shown in Figure 10 in which Figures lOA and B are 
images including respectively an isolated calcification and a microcalcification cluster, 
and Figures IOC and D are the correspondiag gradient maps. It can be seen that the 

1 5 calcifications are more visible in the gradient maps. 

In this embodiment a computed contrast value k is then calculated fi-om the 
gradient map for the image in accoirdance with equation 5 below. This value is set to be 
the average value of the local contrasts plus two standard deviations. This value will be 
subsequently used in the anisotropic difiEusion process and also in the foveal 

20 segmentation process. 



k = meaning) + 2 * std^g) (5) 
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Having calculated the value k an anisotropic difiusion process is applied to the 

4 

clean SMF 9. This involves applying a difEusion tensor s imil ar to that disclosed in 
Weickert, J.: Anisotropic Difiusion in Image Processing. B.B. Teubner, Stuttgart (1998) 
herein incorporated by reference, but using the eigenvalues below:- 



= Ifor gradIJ= 0 



A^ = l- exp 



( \ 
-1 



n 

J 



for\gradl\> 0 



where / is the initial image, 7^ the Gaussian smoolhed unage, k the calculated contrast 

measure and w is a suitably high power such as 8 or 12. It can be seen that where kis 

high, i.e. where the contrast is high, thus indicative of an edge, the value of the 
10 exponential term in is very smaU, thus inhibiting diffusion across the edge. 

Figure 8 illustrates an example of an original image and diffused versions of it 

using ^ = 5, a = 0.6 and fibrstly / = 20 iterations and secondly / = 40 iterations. It can be 

seen that the microcalcification and also noise are more visible in the diffused images. 

Figure 9 A and B illustrate an original mammogram in Figure 9 A .and its surface plot in 
1 5 Figure 9B, together with a diffused SMF of the mammogram in Figure 9C and its 

corresponding surface plot in Figure 9D. In this case the diffusion was conducted with k 

== 15, a = 0.6 and t =5 iterations. 

Figures 5 and 6 also illustrate image diffusion. Figure 5 A illustrates a sample of a 

mammogram containing a large calcijQcation and several artifacts. Figure 5B shows the 
20 result of diffusing the image and the smooth background and calcification can be clearly 

distinguished. In Figure 6, Figure 6 A is an SMF image containing a microcalcification 
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on the left side and a large spot of noise on the lower right side. Figure 6B is a difiiised 
version of Figiire 6A and Figure 6C is the 3-D plot of the SMF image in Figure 6 A. The 
surfece plot shows an extremely noisy appearance and important structures can barely be 
distinguished. However, in Figure 6D the surface of the diflused image of Figmre 6B is 
5 shown and the microcalcification appears as a hill with smoother edges than those of the 
very sharp-edged noise structures in the same image, while the backgromd is sinooth. 
Therefore the important structures can be distinguished more easily. 

The final step in the process illustrated in Figure 4 is the application of an 
adaptive segmentation method, such as the foveal method ejqplained by Heucke et al. 

10 This processing is conducted upon the diffused SMF image 1 1 . A set of mean values of 
the image intensities/values is computed using masks for the inner area, its 
neighbourhood and background. The histogram of the inner surface provides the mean of 
the values in the object (jio), and the histogram of the whole image gives the mean of the 
background values (\x^). The mean of the values in the neighbourhood (ii^) is defined as 

IS the weighted sum of intensities with a suitable scale set for the mask. Then the 
perceivable contrast is:- 




20 



Then a minimum contrast value is computed where M-a" • ji^ +(l-w) • jib where w is a 
sidtable weight between 0 and 1. In practice w=0.923 gives good results. Thus is 
calculated in accordance with equation 7 below:- 
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with Cy, set to practice the value Zf=0.0808 has given good values. Thus 

the segmentation process is based on the value of k, calcidated from the gradient map 
and so adapted to the particular image being processed. Areas where C > are 
5 marked as microcalcifications. 

Figure 1 1 illustrates various original SMF images in Figures 1 1 A, C, E, G and I, 
* with corresponding detection maps in Figures 1 IB^ D, F, H and J. The marked areas are 
those areas where the contrast C is greater than minimum contrast It can be seen, 
therefore, that the microcalcificationare clearly visible m the segmented map 13 
10 illustrated in Figures 1 IB, D, F,.H and J. 

Although the invention has been described in relation to the processing of 
mammograms and in particular the processing of mammograms in the Standard 
Mammogram Format, it should be appreciated that the techniques are applicable to 
mammograms which are not in that format, and also to other images, medical, or non- 
15 medical. 

The invention also extends to a computer program for executing the image 
processing method on a suitably programmed computer system, to a computer readable 
storage medium carrying the computer program, and to an image processing apparatus 
for executing the image processing method. 
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CLAIMS 



1. A method of processing images, comprising applying an anisotropic difiusion process 
to the iinage, the anisotropic difEiision process being adapted in dependenqe upon the 
5 • contrast in the image. 

-2. A method according to claim 1 wherein a diffusion coefiScient in the anisotropic 
difiusion process is adapted in dependence upon the contrast in the iinage. 

10 3. A method according to claim 2 wherein the diffusion coefficient is calculated from 
the local contrast in the image. 

4. A method according to claim 3 wherein the diffusion coefficient is calculated from an 
average value of the local contrast in the image, 

15 

5. A method according to any one of claims 1 to 4 further comprising the steps of 
deriving a Gaussian derivative of the image and applying said anisotropic diffusion 
process to the SMF image. 

20 6. A method of processing images to segment objects in the image from backgroxmd 
comprising applying a foveal segmentation algorithm to the image in which areas of die 
image are assigned to an object if the local contrast is greater than a minimum contrast 
value, wherein the minimum contrast value is defined with respect to the contrast in the 
unage. 

25 

7. A method according to claim 6 wherein the minimum contrast is calculated from an 
average value of the local contrast in the image. 
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8. A method according to claim 6 or 7 wherein the local contrast is calculated from a 
weighted svim of the image intensities in the object and in the image. 

9. A method according to any one of claims 6 to 8 further comprising the steps of 
5 deriving a Gaussian derivative of the image and applying said foveal segmentation 

algorithm to the SMF image. 

10. A method according to claim 4, 7 or 8 wherein the average value of the local 
contrast in the image is calculated over the whole image. 

•10 

1 1 . A method according to any one of claims 1 to 5 further comprising sementing the 
processed image using the foveal segmentation metihod of any one of claims 6 to 10. 

12. A method according to any one of the preceding claims wherein the image is an x- 
15 ray image. 

13. A niethod according to any one of the preceding claims wherein the image is a 
medical image. 

20 14. A method according to any one of the preceding claims wherein the image is a 
. mammogram. 

15. A method according to claim 13 further comprising the steps of identifying areas of 
the processed image as representing microcalcifiications. 
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ABSTRACT 

IMPROVEMENTS IN OR RELATING TO IMAGE PROCESSING 

Image processing method, particularly suitable for processing noisy images such 
as digitised mammograms. An adaptive anisotropic dijBRision processing method is 
described in which the diffusion pariameter is adjusted in accordance with the contrast in 
the image. An adaptive foveal segmentation method is also described in which the 
segmentation parameters are set adaptively in accordance with the contrast in the image. 
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